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This paper describes a LISP function* ERASER, to be used in the 
process of recognising objects by computer* It us a pre-procesabt 
to a program called SEE which finds whole bodies in scenes * A 
short description of SEE and the required data-form for a scene 
ia given* SEE is simulated for five different, a cedes to demonstrate 
the effects of shadows on its operation. The function ERASER is 
explained through its sequence of operation, the heuristics used 
and detailed results for teat cases. Finally, a "how to use it" 
section describes the data required to be on the property lists of 
the vertices in the scene, and the cruft that ERASER puts on these 
p-lists as it operates. 
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Introduction 


This paper describes * LISP function, ERASER, used as an aid 
in the interpreting of a real*-life scene* The scenes considered 
are rand or groupings of rectangular blocks of various sizes* A 
representation of such a scene, useful for manipulation, ie in the 
form of a network of links, showing how, when viewed from seme angle, 
the vertices of the solid objects are connected* The network is a 
mapping of the three-dimensional scene into a two-dimensional repre¬ 
sentation and would be derived previously, by an exterior program, 
from vidisector data* 

ERASER is used to remove vertices and links caused by shadows, 
from the network. With the superfluous data removed, the network 
can be analyzed more successfully by SEE, a function that finds 
bodies in scenes. 

Background Information 

To allow a proper perspective for understanding what ERASER 
does and how it operates, a short description of another program, 
which handles the same data-type, ie necessary* For a more detailed 
discussion, see references (1) and (SJ* 

The present process used for the recognition of objects tay 
vision encompasses three steps t 


2 




1 * A program to convert, tne light intensity information of a 
visual soane, obtained from a vidisoctor, into a network of vortices. 
In a Lisp representation, the network is a list, whose elements pro¬ 
vide information about oath vertex visible to the vidisector, This 
information ■v^juld: bet the name of the vertex, the as— and y-coordi¬ 
nates, and the neighboring vertices (i.e. those vertices in the net¬ 
work. connected to the named vertex)- For example, a cube might be; 

i A (11) [F B) 

B (16) (A E CJ 

C (4 a) (B D) 

D {9 8) (G C E* 

E {6 6} (F D B) 

F (6 1) (A 0 E) 

G {9 3) (F D) ) 

Figure la is a more natural, equivalent way of representing this 
same cube* 

2. A program for processing the input data and determining 
whole be dies, 

i 

3, A matching program to classify the bodies as we 11-understood 
objects for which the system has convenient and easy to manipulate 
models * 

The second step is accomplished by SEE, a ph.D, thesis by 
Adolfo Gusman, in conjunction With a pre-processor. The pre-processOf 
operates on the vertex network and derives information necessary for 
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SEE* Regions are found and labelled, Each vertex is classified as 
being one of eight types fsee Figure 2) — 1>, T, X, K r ARROW, FORK, 
WDLTlj or PEAK . and information ■characteristic to that type is sup¬ 
plied. An L is a vertex linked to two other vertices r A vertex 
linked to three vertices can lie a T, a FORK, or an ARROW, The vertex 
with four neighboring vertices is either an X,or a X+ A vertex not 
fitting one of the above categories is a MULTI, or if the y-coordinafce 
is greater than that of any of the linking vortices the special case 
PEAK is used. For Figure 1: 

Regions t 1, 2, 3 
Vertices of type L: A, C, G 
Vertices of type FORKt E 
Vertices of type ARROWt B, D p F 

Operating on the network and the information supplied by the pre¬ 
processor, SEE makes reasonable guesses concerning which regions 
form bodies. The actual process is to create a network of linked 
regions from the network of linked vertices. The new links, between 
the regions of the scene, are contracted from a systematic analysis 
of the vertices of the scene. Two degrees of association between the 
regions are possible* These are designated H strong" and "weak" links. 
Three functions in SEE merge regions to form nude is according to 
the links that have been formed. GLOBAL requires that regions be con¬ 
nected by at least two strong links to be joined as a nucleus, LOCAL 
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joins regions having one strong link supported by a weak link. A 
single region having only one strong link, which connects it to only 
one nucleus* will be connected with that nucleus by SINGLEBODY, When 
all possible nucleia have been formed, any remaining links are ignored 
and sRR outputs each nucleus as being a body -- listing the regions 
comprising the bodies. 

Except for recently explored and documented weaknesses or special 
problems (see reference (3)), SEE performs well in noise-free situa¬ 
tions. The motivation for ERASER is to enable SEE to handle more 
complicated and realistic situations, namely those having shadows* 

Allowing shadows, the heuristics used by SEE cause improper pro¬ 
cessing of the scene- This is understandable since these heuristics 
are local; specifically, observed characteristics of the transforma¬ 
tions resulting from the mapping of three-space into two—space. A 
FORK is a front-on view of a corner; an ARROW is a side view of a 
corner. The T-type vertex can be either a degenerate view of an edge 
and corner or two regions of one body obscured by another distinct 
body, depending upon: the location of the background, relative to the 
T* the existence of an "opposite"* and "facing 1 * T vertex (such a pair 
of matching Ts are shown in Figure lb), or the slopes of segments 
bounding the regions that the T bounds* An X occurs as the interior 
edge of stacked blocks, in addition, some special case heuristics 
are used which help prevent links that would be formed due to optical 
illusions or point-of-view problems. 
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Although, a heuristic nay Involve a search for a vertex Quite 
removed fan the vertex being analyzed, as in the case of the matching 
Ts, no explicit model or conjecture ie made concerning what type of 
body ie being pieced together, sbe 1 g guesses about the et rget ore 
of material bodies is based cm from one to three edges of a single 
region. 

With BEE irt its present form, there are two obvious types of 
action to be expected for scenes having shadows; 

1, In conservative situations, with insufficient links between 
real body regions and shadow regions, the shadow region ie simply 
ignored and becomes a one-region body. 

2 , When links between shadow and body exist, the shadow 1* 
absorbed into the body and thereby complicates the body,. It makes 
the job of the pattern-matching program more difficult* 

■Simulation of see 

see was simulated for five sample scenes of varying complexity, 
and the above-mentioned actions are observed to have occurred* The 
scenes are shown in Figures 3, 5, 7, 9, and 11* A diagram showing 
three successive stages in the operation of SEE follow the scenes 
(Figures 4, 6, 8, 10, and 12). Each aucb diagram shows all the links 
formed between the regions of the scene, the nucleia formed by the 
merging process, and the results as they might be printed out by SEE, 
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The notation for these diagrams is a a follows? 
the numbers correspond to regione 
solid lines are strong links 
dotted lines are weak links 

the letters are the vertices causing the link. 

The scenes considered in no way caused errors due to degener¬ 
ate viewing conditions or optical illusions. The five cases show 
how the heuristics for solid objects are just unable to cope with 
shadows* The most devastating action caused by the presence of 
shadows is to divide a body into several pieces which would other¬ 
wise have been correctly r*ecognized as continuous* This division 
is caused primarily by the heuristics connected with the T and X 
vertixee (overlapping bodies and interior edges) — very basic 
heuristics which are quite powerful in noises free situations. 

Except for the difficulties created for later processing in 
the vision recognition sequence, the action of connecting a shadow 
with its body is technically correct. I think an indication of 
how three-dimensionally oriented SEE's heuristics are is the fact 
that shadows are merged together into nucleia as SEE tries to pull 
the two-dimensional shadows into a real body* Again, this is 
caused by the X and T heuristics* 

The approach used in the design of ERASER was to categorize 
the characteristics of shadows into a set of heuristics that could 
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be applied to the vertex network prior to SEE. in this way Che 
features caused by shadows, which run counter to the heuristics 
embodied in SEE, are completely removed from the network. 


General Features of ERASER 

ERASER, is to be used just prior to SEE, after the pre-processor 
has examined the simple vertex network and established the neces¬ 
sary vertex information. ERASER is handed a list of all the dark 
regions, the thought being that any shadows present in the scene 
will be contained in this list- Such information about Che regions 
should be readily available from the light gradients given by the 
vidisector. Not all the dark regions arc shadows, however, since 
the face of objects turned away from the light are as dark as 
shadows. And,, same gf the line segmenta bounding a shadow region 
also bound a real body region. Nevertheless, the heuristics in 
ERASER do not remove the real body lines, in most cases. Therefore, 
it would be reasonable to use ERASER as a predicate to decide if a 
given dark region were a shadow or not. 

ERASER does not need to know global information about the 
scene„ for example which body is causing the shadow, or light gradi¬ 
ents across boundaries of the regions. The heuristics are entirely 
local and involve two. or at most three, connected vertices. Two- 
types of heuristics are found in ERASER, grouped as Series One and 
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Series ]Vo. 


Series One is further divided into primary and secon¬ 


dary, 

Th€' biases toward data Chat are built into ERASER are similar 
to those in SEE. With currently no mechanism sat tip to handle 
curved lines,. ERASER assumes the connections in the vertex network 
are straight lines. Also, several of the predicate tests arc 
designed with a right shadow projection id mind. However ( a set 
for left-projected shadows could be made with a choice of which to 
use based on additional information supplied to ERASER, The pro¬ 
jection-dependent tests are only used to effect a smoother operation 
and are not used in the decision to remove a suspected shadow. 

The operational procedure is straightforward. ERASER examines 
the dark regions in the scene,, one by one. Tracing counterclockwise 
around the perimeter of each such region, the sequence vertex 
types encountered is analyzed. The primary heuristics of Series 
One determine if a line segment of the perimeter, connecting two 
vertices, is a shadow line or a part of a body. If believed to be 
a shadow, the segment is marked fgr later removal from the network. 
When all the vertices of a region have been toi side red in this 
manner a new region ia analyzed+ After the last region has been 
examined a function is called which removes the marked segments 
from the vertex network. As shown in Figure 13, vertices change 
type {J goes from an "I” to a straight m l 41 , F goes from "HULTl" 
to "ARROW Ir ) or disappear (II, I — shadow vertices of type ”L"} * 
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[U (X Y) IF I}) 

[I IX Yl (H J)} 
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, j fine qive an Lmmedrate 

The secondary heurratios of Ser^-> unt 4 

ava l„a*io» of the primary l-ri-tlo.. *»- th. primary b-'K'*" 
are finished and before the nett vertex of the perimeter 1. hreught 
into CO naideration, ERASE* is looking at two lire eegmente (see 
Pigare 12.1a). M this point the secondary heuristics are called 
in to reconsider the decision made regarding the Pf-ions line seg¬ 
ment. This decision is either accepted or reversed, based upon 
the current line segment (two vertices) just examined and the deci¬ 
sion regarding it. This set of rules is most useful for removing 
the leading edge of a shadow (segment (F H) in Figure 13) or for 
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replacing part of a T that has been too quickly removed (segment 
(AN AK) in Figure 11]* Modification of the vertex network tends 
to be an irreversible procedure and it is due to the secondary 
heuristic? that the segments are only marked, and not removed, 
until all the regions of a scone have been examined 

To effect the operation of the Series One heuristics, a set 
of pattern-matching predicates were derived, based on two facta 
about shadows that allow their quick and easy recognition* 

1, the abundance of La, Ts, and Xs bounding shadow regions, 

2. the pattern of the occurrence of those three types of 
vertices in conjunction with the occurrence of the other possible 
types of vertices bounding any particular region. 

These two facts are relevant in the following ways. First, 
the three vertex types mentioned are those caused most often by 
shadows. The L is the projection of a corner of a block located 
above the plane of the L- Normal intersection? of shadows with 
bodies appear as Ts or Xs, e+g. whan a body blocks the view, or 
a shadow or one body falls on two faces of another body. Second, 
the typical shadow has the following sequence of vertices a? part 
of its perimeter fsee Figure 13] j ARROW - L - L - T- Of course, 
the viewing angle can, and does, affect the actual vertices of the 
shadow, with tho exception of the Ls and Xs (other than obscuring 
them). Regular deviations from the typical case are to: replace 
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the T with an X or a K t insert a aerie* of Ts and/or Xs between 
the LS? or .replace the ARROW with practically any type of vertex. 
When con aider ihg two vertices, qne of which ia of the type T, K, 
or X, it is necessary to Knew if the other vertex is in the con¬ 
tinuing line of this T, K, or X* For example, in Figure 2, ver¬ 
tices A and C are in the continuing line of the "T" vertex E; ver¬ 
tices D a nd E are in the continuing line of "X' 1 vertex C. A line 
segment that forms part of this continuing line cannot he a shadow 
line and is therefore never removed. Or, if removed by Series 
One/primary,, it will be replaced by Series One/ secondary, This 
rule is the basis for determining shadows and is specifically what 
the predicate tests for. 

the Series Two heuristics are an attempt to make the analysis 
of the dark regions more realistic, i-e. more like what I imagine 
human analysis to be. Taking a hint from the way the X heuristic 
of SEE tends to pull shadows together, ERASER, through Series Two, 
tries to pieoe together dark regions to form a continuous shadow. 
As a result of these two types of heuristics the operation of 
ERASER is quite realistic. 

An examplei ERASER follows the shadow of the upright body 
in Figure 9 from the supporting surface, up the face of the solid 
object f across the top of the object, and then looks for the con¬ 
tinuation of the shadow on the suporting surface, behind the long 
body. Being able to piece together the dark regions in this way 
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seetns ta yield a potentially valuable piece of information abut the 
scene itself* One additional device could be added to improve this 
general semantic processing. ERASER should recognize that vertex All 
presents an unnatural change of direction and that such changes of 
direction can be attributed to intersecting shadows. It would then 
jump up to vertex AV to continue its analysis, returning laten to All 
as the starting point for the second shadow* To be able to do this 
■efficiently would require either more global heuristics or some type 
of pre-ERASER processing of the scene. 


HeUT.i3tj.cg of ERASER 

The primary and secondary heuristics embodied in the series One 
group can be summarized as follows: ^notation — CV ■ ,l nm <l , the cur¬ 
rent vertex is type "nm"; PV * "irn", the previous vertex is type "nm") 

Prints ry 
i- CV - L 

a- PV - FORK, ARROW, PEAK, K QT MULTI; no action 

j- 

b, PV =*■ L; remove the line 

c_ PV — T and CV is not in the continuing line of the T; remove 

i 

the line 
2* cV ** T 

a- PV *= X on T and neither vertex ie in the continuing lino of 
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of the other? remove the line 


fcu tV = L and PV is not in the continuing line Of CV? remove 
the line 

C* PV - FORK, ARROW t PEAK* K or MULTI; ilD action 
3. CV * X 


a. PV * any and PV is not in the continuing line of OVf remove 
the line 

4, CV =* K i s tr ea t ad a a a spec La 1 case d f CV ** T ahovs 

5. CV = FORK, ARROW, PEAK or MULTI } no action 


Secondary . 

6. If pv = T and the previous line was removed and the current 

vertex ie in the continuing line of the PV? replace the previous 
line 

1. If PV = L and the previous line was not removed and the current 
line has been removed? remove the previous line 

6* If PV ■ X and CV is not in the continuing line of the PV and the 
current line has not already been removed; remove the current 
line* 


The two heuristics of Series Two can he stated as follows; 

1* When ERASER discovers the current vertex to be an X (Figure 
12.1b, vertex A), rather than turn left as it does for, say, a MULTI 
(Figure 12+lb, vertex C) it continues on across the centinning line 
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of the X and adds the perimeter of the region left of segment All to 
the perimeter of the region currently being analyzed. 

2. A 1 vertex (figure 12.1c, vertex A) causes ERASER to 
look right along the continuing line of vertex A for a T whose Up¬ 
right is oriented opposite to that of the current vartex (as shown 
by vertex B). Finding B, the region above AB and having segment AB 
as part of its perimeter is added to the current region. However, 
had E been the first vertex to the right, ERASER would have proceeded 
normally to vertex C. 

As ERASER operates it documents which heuristic it invokes for 
a decision. The best way to demonstrate and explain these Series One 
and Two heuristics, as listed above, is to examine the printout from 
ERASER for some scenes. 

The documentation ERASER yields is in three parts* 

1. After printing its own name all the dark regions handed to ERASER 
are Hated with: 

a* a name, vis* SEADOVn, where n is a number; 

b + a list of the vertices bounding the region, given in counter¬ 
clockwise order, 

2, When each dark region is considered the work HEURISTIC (the main 
function) is printed followed by the name of the region being 
considered* immediately after this will be three types of infor¬ 
mation: 
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a. (vl Tl v2 t 2) -- vl and v2 are the names of the vertices 
being analysed; Tl and T2 are the types o£ the previous 
vertex, vl, and the current vertex, v2, respectively* 

b. (Tl “—^ T2) — Tl and T2 are the same vertex types as in 
(a); however, this basic form will be in one of four formats. 
It shows whether a Primary or Secondary heuristic was applied 
(refer to Figure 12.1a}. 

i. Tl====>T2 e delete current line, primary (1)^(5) 

ii. -^******Tl ? restore previous line, secondary (6) 

iii. *ir****>Ti ■ delete previous line, secondary (7) 
iv + pi*+** delete current line, secondary (0) 

C* { (CALL RGNl t) Vl El v 2 ) This is printed when a Series Two 
heuristic is used. If t is X, then (i) was ueedf if t ia T, 
(ii) was used? vl and v2 are two vertices in the perimeter 
of the region being added to the current shadow. (RGNl is 
the name of the function that actually does the merging.) 

3. The final part ie a lieting of the results and is selfexplanatory 
Figure 13 shows the before and after plot of a simple scent, Thu 
ERASER printout is presented in Figure 14. For SIWDCWl, the real 
body region, the no-action heuristica are (from the above charts): 
l(a);*^2(b) (since the L is in the continuing line of the T) f 5r 5? 5 
SHAD 0 &f 2 is actually a shadow and three line segments are removed by 
these heuristics: 1(a)? 1(b) e 7j 2(b) 1(c); 5. Examples of the 
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Series Two heuristics can be found in the printouts for both Figures 
9 and 11* Figures 15 through 24 give the before/after plots of the 
five scenes shown earlier along with their diagnostic. The plots 
Show the SUCOSSS Of ERASER; SEE WOuld now have no problem?+ 


All That Glitters* *, 

The debugging of ERASER led to a couple of thoughts about when 
it will lose* Very dark blocks are definitely bad news. A light 
block and a dark block side by side look locally like a light block 
with a shadow on it. For such a situation shown in Figure 2 5a t if 
ERASER stumbled onto the boundary between the blocks {segments AB 
and Ed , it would be removed. The snorted block of Figure 21 shows 
the shadow consideration of this problem. I presently see no method 
for locally deciding this problem. 

When a couple or three vertices are marked by the primary heuris¬ 
tics for removal and the continuing fine rule remains satisfied the 
line segments are quite dead- Various combinations of obscuring 
bodies will have the lethal sequence of vertex type? if there are 
not sufficient shadows to break up the perimeters of the regions. In 
Figure 5 r segment TU shows where a T-T sequence should be eliminated 
and segments AB and CD in Figure 25b show where the same sequence 
should not be removed* This again seems like a problem requiring a 
global solution. 
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How to tJse Tt 


The complete ERASER is contained on the file named "ERASER n" 
where n is the highest integer found* Figure 2& la a printout of 
all the information about the scene of Figure 3 that is needed for 
successful operation of ERASER. Each vertex is named and hag been 
given an x and y coordinate, a TYPE,, a TYPE*, and a NEIGHBORS 
list * These properties are explained in the paper by Guzman, The 
argument to ERASER is a list of line segments* A segment is a line 
from the perimeter of one dark region, and is represented toy the 
list of its two end points. ERASER expands the line segment into 
the complete perimeter of the dark region and begins computation. 

If a vertex is the endpoint of a segment that is to be removed, 
the other endpoint of the segment Is put on its property list under 
the indicator KILL* At segment removal time this KILL vertex ie 
removed from the NEIGHBORS list. 
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